Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Use Nix in CI where possible #5394

Merged

Conversation

I-Al-Istannen
Copy link
Collaborator

I decided I hadn't inflicted enough pain on myself lately, so I spent a while trying to convert our CI pipeline to nix.

Benefits:

  • It uses Nix
  • If maven had proper lockfiles, we could nicely cache maven dependencies (it does not, so we can't)
  • Running things locally and in the CI should produce exactly the same output (modulo maven dependencies…)
  • We get some caching for free using the magic-nix-cache action (e.g. the z3 artifact)
  • All commands run in the CI are available locally, you can enter a dev environment shell that is an exact copy of the CI with a single command and execute things without needing github actions or act
  • We get rid of all the weird setup actions

Most of them come down to potentially better performance, but most importantly, exact reproducibility of the CI environment (modulo tokens) locally.

Downsides:

  • It uses Nix
  • It only works on Linux / Mac. This is only a problem for our Test matrix, as the rest runs on ubuntu-latest anyways.
  • Hydra does not build z3 with java bindings, so the first CI run will spend half an hour building z3 from source, before it is cached
  • I needed to package two libraries (semver and jreleaser) myself as they are not in nixpkgs. Jreleaser's build process has ample opportunities for improvement: It fails tests, or jdk downloads, or something else everytime I run it — ON ITS RELEASE commit. So now we package the jreleaser binary instead.
    Packaging things ourselves means we are responsible for updates though
  • We can't make use of quite some automatic caching because maven's dependency resolution is a mess and you can not really verify anything locally, without making a network request to check what the repository thinks. This is verboten in nix builds, as it is impure.
  • I have basically no idea what Nix is, so there are probably a lot of things that can be improved

Other questions:

  • Do we want this at all?
  • Do we want to check our flake in every job or just once? We shouldn't forget doing it, but doing it multiple times doesn't really help.

@I-Al-Istannen I-Al-Istannen force-pushed the chore/flakey-flakey-rise-and-shine branch from 2b72cf1 to 8499d5d Compare August 24, 2023 22:59
@MartinWitt
Copy link
Collaborator

your change to the release script could conflict with #5369.

@I-Al-Istannen
Copy link
Collaborator Author

It should not conflict, those are new workflows. But we would also want to translate those, I imagine.

@MartinWitt
Copy link
Collaborator

Yes, this was meant. Could you do this and we drop my PR?

@I-Al-Istannen I-Al-Istannen force-pushed the chore/flakey-flakey-rise-and-shine branch 2 times, most recently from cda78b1 to 916ca63 Compare August 26, 2023 20:16
@I-Al-Istannen
Copy link
Collaborator Author

Done, I think :)

@I-Al-Istannen I-Al-Istannen force-pushed the chore/flakey-flakey-rise-and-shine branch from 916ca63 to f2f9d21 Compare August 26, 2023 20:52
@monperrus
Copy link
Collaborator

very much in favor of Nix in CI, that should be long-term reproducible than Docker (claim that I would love to substantiate with a link to a thesis or paper).

Copy link
Collaborator

@MartinWitt MartinWitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit scared about the flake but worth a try. So in the future, we write bash scripts for advanced stuff and call them from nix?

.github/workflows/jreleaser.yml Show resolved Hide resolved
.github/workflows/tests.yml Show resolved Hide resolved
@MartinWitt
Copy link
Collaborator

@monperrus could you disable the beta release workflow in the spoon-deploy repo, see https://docs.github.com/en/actions/using-workflows/disabling-and-enabling-a-workflow?

@I-Al-Istannen I-Al-Istannen force-pushed the chore/flakey-flakey-rise-and-shine branch from e59f167 to 73581e5 Compare August 27, 2023 13:16
@I-Al-Istannen I-Al-Istannen force-pushed the chore/flakey-flakey-rise-and-shine branch from 73581e5 to 6a350b2 Compare August 27, 2023 13:29
@I-Al-Istannen
Copy link
Collaborator Author

So in the future, we write bash scripts for advanced stuff and call them from nix?

There are many ways to do things. I currently added small commands as dev commands (writeShellBin in the flake.nix) and larger ones got their own bash script in chore (following established precedent there). The idea is that we use a coherent language that works everywhere and does not depend on github (actions) wherever possible. This allows people to run things locally and the more inputs we can track in Nix, the easier it is to reproduce behaviour locally. We could also use python or something else, if you prefer that. bash is just quick'n'dirty to write, but I am a Linux user :P Maybe you have a nice windows idea, though the Nix env only works on Linux and Mac.

I also see the irony with #5333, but we got caching for free and eliminated random setup commands.
We could also split up the script in a few separate scripts that run in parallel jobs:

  1. Add a job that builds spoon and stashes the resulting maven repository in a new job-local cache
  2. Use a matrix job that depends on the first job to spawn the factored scripts in parallel

@I-Al-Istannen I-Al-Istannen force-pushed the chore/flakey-flakey-rise-and-shine branch 5 times, most recently from e36b1fa to 0d97462 Compare August 27, 2023 16:58
@I-Al-Istannen I-Al-Istannen force-pushed the chore/flakey-flakey-rise-and-shine branch from 0d97462 to 126bba5 Compare August 27, 2023 17:02
Factor out common setup into its own reused workflow
@I-Al-Istannen I-Al-Istannen force-pushed the chore/flakey-flakey-rise-and-shine branch from 126bba5 to 6c22191 Compare August 27, 2023 17:25
@monperrus
Copy link
Collaborator

@monperrus could you disable the beta release workflow in the spoon-deploy repo, see https://docs.github.com/en/actions/using-workflows/disabling-and-enabling-a-workflow?

done

@I-Al-Istannen I-Al-Istannen force-pushed the chore/flakey-flakey-rise-and-shine branch from 6c22191 to 0dbed0c Compare August 28, 2023 14:07
@I-Al-Istannen I-Al-Istannen marked this pull request as ready for review August 28, 2023 15:08
@I-Al-Istannen I-Al-Istannen requested a review from SirYwell August 28, 2023 15:09
.github/workflows/update-flake.yml Outdated Show resolved Hide resolved
.github/workflows/tests.yml Show resolved Hide resolved
@I-Al-Istannen I-Al-Istannen force-pushed the chore/flakey-flakey-rise-and-shine branch from 13dc48e to fecf6be Compare August 28, 2023 19:15
@I-Al-Istannen I-Al-Istannen requested a review from SirYwell August 29, 2023 10:50
Copy link
Collaborator

@SirYwell SirYwell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Let's go on this adventure then.

@MartinWitt MartinWitt merged commit fd2c6ab into INRIA:master Aug 29, 2023
11 checks passed
@MartinWitt
Copy link
Collaborator

Thanks @I-Al-Istannen. Let's do the first test at 0 UTC and watch the snapshot release?

@I-Al-Istannen I-Al-Istannen deleted the chore/flakey-flakey-rise-and-shine branch August 29, 2023 16:10
@monperrus
Copy link
Collaborator

super cool thanks @I-Al-Istannen

let's document this a bit? could you edit #5415 accordingly?

@aalmiray
Copy link

  • I needed to package two libraries (semver and jreleaser) myself as they are not in nixpkgs. Jreleaser's build process has ample opportunities for improvement: It fails tests, or jdk downloads, or something else everytime I run it — ON ITS RELEASE commit. So now we package the jreleaser binary instead.
    Packaging things ourselves means we are responsible for updates though

Nix support is in the roadmap, both for JReleaser itself and every other consumer. jreleaser/jreleaser#1032

@I-Al-Istannen
Copy link
Collaborator Author

I-Al-Istannen commented Aug 30, 2023

@aalmiray

I think I will have a look at nicer nix maven builds in the upcoming future too (it ties in quite nicely with https://github.com/chains-project/maven-lockfile and gradle supports dependency lockfiles for years now). As far as I can tell, packaging things for maven currently is a bit annoying and needs to rely on fixed-output derivations, which will need updates for very minor dependency update. But my Nix game is not strong at all :P

For the moment the binary packaging I built in this PR seems to work just fine (it is basically a minimal nix version of your java download script from the jreleaser-action). Source-based packaging is a bit annoying in CI, unless you can offload the binary cache to hydra :P We build z3 from source, but it is cached in GitHub actions — still takes ~30 minutes for the initial CI run though, until the package is in the GitHub cache.

I was a bit surprised that I did not manage to even build Jreleaser manually (I first tried to have a from-source derivation). I have no idea how you do it, but everything I tried to build it failed, even on the release commit. It's probably me being stupid, but maybe something on your side is a tad weird too?

@aalmiray
Copy link

I was a bit surprised that I did not manage to even build Jreleaser manually (I first tried to have a from-source derivation). I have no idea how you do it, but everything I tried to build it failed, even on the release commit. It's probably me being stupid, but maybe something on your side is a tad weird too?

Given that JReleaser is a Java project built with a very specific version of Gradle I'd suspect it may not be possible to build it from a Nix environment without network access.

It should be enough to run gradlew build with Java 11 or greater to build the project. Note that the given Gradle wrapper must be used, no other wrapper or command will do.

@I-Al-Istannen
Copy link
Collaborator Author

Given that JReleaser is a Java project built with a very specific version of Gradle I'd suspect it may not be possible to build it from a Nix environment without network access.

You can access the internet if you know the hash of the result (which only helps if you have useful lockfiles). Nix packages gradle 6.9.4, maybe that would work as well. But I am happy with just taking your binary at face value right now :)


It should be enough to run gradlew build with Java 11 or greater to build the project. Note that the given Gradle wrapper must be used, no other wrapper or command will do.

I tried it on my host as well, but didn't notice that my system locale was screwing things over there… Classic :P I made a tiny PR with a potential fix.

Thank you for the help and enjoy your evening :)

@aalmiray
Copy link

Thank you for the PR, I;ll have a look at it shortly.

FWIW JReleaser's binaries come with SBOMs, SHAs, digitial signatures, and SLSA provenance. Thus, allowing multiple ways to verify them.

@MartinWitt
Copy link
Collaborator

Given the test list:

  • Snapshot Releases
  • Beta Releases
  • Real Maven Central release

Two are left. Do you want to wait for a beta release, or shall we try patch release first?

@MartinWitt
Copy link
Collaborator

Okay we did patch release test and it works. First spoon release with the new nix CI 🎉 https://github.com/INRIA/spoon/tree/v10.4.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants